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ABSTRACT 

During the 2002-2003 academic year, six roundtables were held 
to explore uses of the data from the National Survey of Student Engagement 
(NSSE) for improvement of student learning. This paper describes the 
roundtable discussion held in St. Louis, Missouri, in March 2003. 

Participants were student affairs administrators from a range of institutions 
across the United States with representation from nearly every sector of 
higher education. Participants focused on the ways student affairs 
administrators have used data from the National Survey of Student Engagement 
(NSSE) on their campuses. Specific topics were: (1) the impact of NSSE data 

use; (2) ways that data are used by participants to meet divisional missions; 
(3) ways in which student affairs staff can interpret NSSE data for students 
and other members of the community; (4) possible applications of the data 
that are desirable but that have not yet been attempted; and (5) tracking 
access and use of NSSE data by constituents. Important themes and issues that 
emerged from the roundtable include key institutional uses of NSSE data, 
creative uses of data by student affairs administrators, and ideas for 
improving data collection and use. (Author/SLD) 
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Student Affairs Administrators: Building Collaborations with Students 
and Academic Affairs for Institutional Improvement 

By Adrianna Kezar 
University of Maryland College Park 

During the 2002-03 academic year, AAHE and NSSE are conducting six roundtables to explore 
uses of NSSE data for the improvement of student learning. A roundtable discussion at the Na- 
tional Association of Student Personnel Administrators* conference in St. Louis in March 2003 in- 
volved student affairs administrators drawn from a range of institutions across the country and 
with representation from nearly every sector of higher education. Participants focused on the ways 
student affairs administrators have used data from the National Survey of Student Engagement 
(NSSE) on their campuses. Specific questions examined: 

■ impact of NSSE data use, 

■ ways that data are used by participants to meet divisional mission, 

■ ways in which student affairs staff can interpret NSSE data for students and other members 
of the community, 

■ possible applications of the data that are desirable but that have not yet been attempted, 
and 

■ tracking access and use of NSSE data by constituents. 

Important themes and issues that emerged from the roundtable include key institutional uses of 
NSSE data, creative uses of data by student affairs administrators, and ideas for improving data 
collection and use. 




Key Institutional Uses of NSSE data 

Sharing results by division or college: Many campuses use NSSE data at the local level. Institutional research 
offices are provided with specific information for the deans or administrators by college or division, and the data 
are used to initiate discussion about ways to improve the institution. Many campuses realize that using data first 
within units may be more meaningful than trying to apply it institution-wide. One student affairs administrator 
commented about beginning with unit analysis of results: 

This unit use allows colleges and divisions to focus on the results in their area. Many people have trouble 
thinking about the overall institution, but can easily think about ways to make changes within their own 
unit. Certainly this is only the beginning and we need to think about the entire institution as well. But this 
is a way to begin the dialogue, in a way that engages people initially with information about their own unit. 

Hosting unit roundtables to create dialogue: Some campuses host roundtables to actively engage each unit and 
college in a discussion of the data. Many focus group participants mentioned the dangers of providing comparative 
data. If information is shared by college or unit, the individual group should only receive its own data and not be 
compared to other schools and colleges. When comparison data have been used, it has sometimes created animos- 
ity toward the instrument and its message. 

Conducting strategic planning and budget reallocation: NSSE data are most commonly used for strategic 

planning. Many student affairs administrators noted that a major initiative based on NSSE data was either active 
learning or collaboration between academic and student affairs. These matters had not been considered in previous 
planning efforts. The NSSE data provided support for making these a priority. Also, units and colleges may decide 
to share data, which they use to inform their planning. One administrator commented: “The data are really being 
used to create programmatic direction. Since I have been on my campus, I have never before seen this attention to 
data for decision making.” 

Campuses also use planning to reconfigure historic budget allocations. One administrator noted: “We hired 
new staff this last year, redirected funding to the counseling center and residence halls based on the NSSE results.” 
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Reallocation of funding is based pardy on information about areas that have received low benchmarks and need 
additional support. Additional funding was given in one instance to first-year programs and in others to writing 
across the curriculum, which has been documented on several campuses to increase learning. 

Providing evidence for accreditation: Several accreditors that now require assessment data accept NSSE as a 
source for understanding student experience and learning. In addition, the data can be used to create benchmarks 
of institutional performance and to target improvement. When accreditors return to institutions, NSSE data col- 
lected over consecutive years provide a way to assess whether campuses have successfully created a better learning 
environment for students. 

Using the data within key institutional teams: Many campuses now have teams to address cross-functional 
issues such as retention, academic success, the first-year experience, general education reform, and the like. The 
NSSE data provide these teams with evidence on which to base campus policies and practices. These teams are 
designed to draw on expertise across campus and to develop solutions to complex problems and issues. The addi- 
tional data have been a key tool in facilitating the work of these teams. 

Involving Boards: Some campuses use the data to create broad policy changes. Sharing NSSE data with Boards 
can help to facilitate dialogue around institutional changes that might not otherwise occur. Although this sharing 
needs to be approached with care so that the data are not used out of context, many campuses find allies with 
Boards that appreciate that the data are instrumental in making decisions about institutional direction. 

Creative Uses of NSSE Data: The Contribution of Student Affairs Administrators 

Blurring organizational lines: The NSSE data have helped to build bridges between academic and student affairs 
divisions and to blur organizational boundaries. The benchmarks demonstrate the shared responsibility for learn- 
ing. As campus groups review the data, they realize that all divisions need to work together to create a successful 
learning environment for students. As one administrator noted: “We are now renovating space and hiring new 
people, but built on a shared model of working together. Space and people are no longer a part of exclusively one 
division or another.” This administrator, like others, welcomed the blurring of organizational lines. 

Re-examining the first-year experience: Most student affairs administrators are focused on ways to improve the 
first-year experience since this time is critical to retention and success. For the past decade, institutions have in- 
vested in the development of first-year seminars, more extensive orientations, and welcome programs to facilitate 
the transition to college. Since NSSE is conducted at the end of the first year, it provides important data about 
whether these new efforts work as well as other institutional processes that might need to be examined. Student 
affairs administrators use the data to examine their success with first-year students and to develop ideas for chang- 
ing programs and practices that appear ineffective. 

Conducting student focus groups and interviews: Student affairs administrators have developed creative ways 
to involve students in data interpretation and in the creation of qualitative data to supplement the more global find- 
ings of the quantitative survey. One student affairs administrator described how administrators have teamed with 
students to make the data even more meaningful: 

We used the data as a starting point. We discussed the results of the data with student leaders and organi- 
zations and asked them to go out and interview other students and hold focus groups in order to see if the 
results resonated with students’ experience. We also asked them to explain the results and provide more 
depth. For example, if academic challenge is low: In what areas? Do they have ideas for improvement? 

This can help us move from data that point out problems to solutions for institutional problems. 

This suggestion resonated with many student affairs administrators who agreed that this was a very useful way to 
secure student involvement in the change process. Another representative said: “We were really bothered by our 
results and many people wanted to discount them. By giving the data to students and having them agree that the 
data reflected their experiences, it provided more power to the data and our efforts to create institutional change.” 
Student affairs administrators thought that students should be aware of the NSSE results and should be partners in 
interpreting and using the data. 

Holding retreats: Student affairs divisions were likely to use the data for planning and innovation. A successful 
way to accomplish this task is through retreats. One administrator described the way NSSE data were used at a 
recent student affairs divisional retreat: 

We recently met as a whole division to examine our performance. The NSSE data were distributed. 

We obtained some simple tables from the institutional research offices. The overall data can be a bit 
overwhelming, I began by reviewing the data and asking for people to comment and interpret the 





Issue 3 — June 2003 



Page 3 



data. After some dialogue about the meaning of the data, we began to discuss what we might do to 
improve the learning environment for students. 

Retreats and extended conversations were meaningful ways to engage the data since these settings tend to encom- 
pass the critical aspects of information sharing, dialogue, examination of performance, review of planning and pro- 
posed innovation. Although many campuses engage in one or more of these planning activities, campuses that 
have retreats engage in many if not all of these key institutional processes. 

Bringing “individual points of light” together: Administrators noted that the survey helped them to identify 
what they termed “individual points of light” - key programs that helped engage students as well as the way these 
programs worked in concert. TFe survey helped them to see the way learning communities, undergraduate research 
programs, and first-year seminars are all combined to create an engaged experience for students. This analysis helps 
administrators decide on the best programs to support and to link into a common strategy. Previously seen as dis- 
jointed programs, they are now seen as part of a coordinated agenda to create engagement. 

Rethinking mission statements: For some student affairs administrators, the NSSE data have resulted in a fun- 
damental rethinking of their work. One campus noted that they changed their divisional mission statement as a 
result of discussing the NSSE data. The student affairs mission statement was not as focused on learning as the 
administrator thought it should have been. Although many units may not change their mission statements, NSSE 
data can be a way to examine roles and responsibilities toward the goal of student learning. 

Ideas for Improving Data Use and Collection 

Using an Assessment Council: On larger campuses that have a variety of assessment efforts, assessment councils 
can coordinate data collection, interpretation, and use. These councils review surveys and approve their use, exam- 
ine survey data together to plan for the student affairs divisions as well as campus-wide planning efforts, and struc- 
ture discussion around the data. The councils also unify the campuses around the meaning of survey data. Before 
the councils developed, each division would get copies of the data and develop their own interpretation and use. 
The council enabled the divisions to compare interpretations and clarify misinterpretations. NSSE data use has 
been improved by the development of assessment councils. 

Collecting data over time: Many campuses noted that having a track record of using NSSE data is important for 
the data to have meaning. Several campuses are in their second or third round of data collection. Although for 
most campuses it is too early to see change in scores, they are hopeful about improvement through changes made 
to respond to concerns noted by students. All campuses acknowledged the importance of collecting data on an 
ongoing basis. Although collecting data for benchmarking is important, it was more significant for these adminis- 
trators to chart progress and change over time. 

Building a story or case: Several people referred to the ongoing data collection process as “building a story or a 
case.” Campuses were better able to interpret the data when they were discussed comprehensively as a story or case 
about the campus. People struggled to understand individual sets of data described out of context. Comparing re- 
sults over several years and interpreting how an institution evolves builds an ongoing story of institutional perform- 
ance. Especially when campuses are about to engage in radical change, building a case for the change is necessary 
and this is where NSSE data play a key role. A representative from one campus explained: 

We wanted to reduce class size. It has been a problem for years. We presented the data at the faculty 
senate and used them to argue for more faculty lines. The faculty senate voted to approve redirection of 
funds and then it went to the administration. We were able to get the reduction in class size and new 
faculty lines. In previous years, we may not have been able to accomplish this without the data. 

Using multiple data sources: Student affairs administrators use other national datasets, such as the Cooperative 
Institutional Research Project (CIRP) and College Student Experience Questionnaire (CSEQ), in conjunction with 
NSSE in order to understand the student experience. CIRP provides information about student expectations of 
college, and NSSE examines their actual college experience. Also, several institutions have developed surveys that 
examine campus climate or student satisfaction. These additional sources of data help to sketch a complex and nu- 
anced portrait. 

Maintaining response rate: With an overload of surveys, students no longer complete surveys at the same rate as 
in previous years. Collecting multiple forms of data carries the risk that students will feel overburdened, resulting in 
lower response rates. Some institutions are examining and consolidating their data collection efforts; others use on- 
line surveys to improve response rates. Several student affairs administrators described the importance of incen- 
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tives. Asking the bookstore to donate a gift certificate costs the institution nothing and generates favorable public- 
ity for the bookstore. Improvement in the institution’s response rate ensures more valid data. 

Breaking down the data into small, more focused tables: One administrator has the institutional research of- 
fice run specific and simple tables for discussion at staff retreats. Others in the focus group expressed a need for 
custom data tables to serve the requirements of particular constituents. A faculty meeting, deans’ council, student 
leaders’ meeting, or divisional retreat might all require different types of data tables. Each group needs accessible 
data. 

Adding additional questions to the survey: A few campuses represented at the focus group have added addi- 
tional questions that capture issues of concern to them. One participant, for example, noted the special conditions 
of a women’s college environment that benefited from customizing data by inserting additional questions. 

Hearing interpretation from people in like roles: Student affairs administrators agreed that faculty were more 
compelled to engage with and believe the data if the data were interpreted by other faculty. Dialogues, roundtables, 
and other forums for discussing the data were more successful when facilitated by faculty. Student affairs adminis- 
trators who work closely with faculty on programming acknowledged that faculty partners serve as a link to their 
colleagues in discussion about NSSE results. The level of receptivity depended on who delivered the data. 



Concluding Remarks 



Participants concluded that student affairs administrators use NSSE data to guide decision making, even if data 
were often difficult to come by. They hope that data will become more accessible to professionals in their role. 
They most enjoyed using NSSE data to facilitate campus conversations. However, usage depended entirely on the 
institutional culture. Participants stressed that engagement was based on institutional mission. Individuals at large 
campuses needed reassurance that their institutions were not expected to have the same understanding of engage- 
ment as a small liberal arts college. The data could help to steer the discussion towards appropriate engagement, 
based on the institutional type and culture. 

This focus group recommended that institutional research offices and campus leadership should ensure the 
NSSE data were distributed to student affairs administrators as they had important ideas about usage and planning. 
They reiterated that student affairs should not use the data in isolation of academic affairs, for example; overall 
campus planning was more useful if it was based on collaborative efforts. Collaboration was the most important 
principle for creating innovation, prompting improvement, and increasing institutional performance. 
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